Skip to content

fix(cache): keep lightweight refs after a refresh instead of retaining Episode objects - #105

Open
nickwolf wants to merge 6 commits into
JourneyDocker:mainfrom
nickwolf:fix/bound-refresh-change-lists
Open

fix(cache): keep lightweight refs after a refresh instead of retaining Episode objects#105
nickwolf wants to merge 6 commits into
JourneyDocker:mainfrom
nickwolf:fix/bound-refresh-change-lists

Conversation

@nickwolf

@nickwolf nickwolf commented Jul 25, 2026

Copy link
Copy Markdown

Rebased onto main now that #102 has landed, and out of draft. The diff here is just the follow-up work.

Follow-up to #102, as promised there.

#102 stopped the refresh from building one giant list of episodes, but the added and updated lists it hands back still hold full Episode objects, so the peak just moves from the fetch to whatever holds those lists. On my library (65,926 episodes) I forced the pathological case, every episode marked updated, and the process peaked at 129.4 MiB against a 1 GB limit. Before, it was going over 1024 MiB and getting OOM-killed. The refs cost 242 bytes each.

The change is a frozen EpisodeRef dataclass carrying just the fields the post-refresh consumers actually read, plus the scheduler and status handlers updated to use it.

There's a second fix in here that I think is the more interesting one. Episodes that come back from a section search have librarySectionTitle set only on the MediaContainer, not on each Video, so the attribute is None on every item. plexapi treats None on a partial object as "go fetch the real value" (base.py:657, if value not in (None, []): return value), which means reading that one field reloads the entire item over the network. That was 65,925 extra metadata requests and about 19 minutes per refresh, all inside self._lock, and any one of them timing out took down the whole run. iter_episodes now stamps section.title onto each episode before yielding, the same thing plexapi does at video.py:1318. Measured 200 GETs down to 0, peak 165.9 to 129.4 MiB, runtime about 20 min down to 6m22s. The existing code pays this same cost today in the scheduler's item.librarySectionTitle.

Happy to split that stamp out into its own PR against #102 if you'd rather keep the two separate. It stands on its own and it's the smaller of the two.

Three things you should know before merging:

  • This doesn't fix the notification storm. That's a separate problem and I haven't touched it.
  • Log strings change for shows with an empty title. The refs carry the stamped section title, so a couple of messages read slightly differently than before.
  • A show that can't be fetched is now skipped rather than fatal. I think that's the better behaviour, but it is a behaviour change and you may disagree.

I've been running this on my own setup since 16 July. Nine days in, the container's memory high-water is 180 MB against a 1 GB limit, with no restarts.

Tests: there are 9 covering EpisodeRef and iter_episodes, using fakes rather than a live server. The repo's old suite needed a real Plex server and a hardcoded episode count, and it died as collateral in 709cc76. I didn't try to revive it. If you'd rather not carry these, say so and I'll drop them.

Does this look like the right approach to you?

@JourneyOver

Copy link
Copy Markdown
Member

It honestly looks all fine to me, and I don't mind keeping the test stuff as I was planning on eventually adding in tests at some point anyways so might as well get a couple tests going.

You will need to fix up the merge conflicts whenever you get a chance ^^

nickwolf added 6 commits July 28, 2026 11:55
…oad per episode

Episodes returned by a section search carry librarySectionTitle only on the
MediaContainer, so the attribute is None on every item. Reading it made plexapi
reload the full item over the network once per episode: 65,925 requests and
~19 minutes on a full-library refresh, all while holding the cache lock, and any
one of them timing out aborted the refresh.
@nickwolf
nickwolf force-pushed the fix/bound-refresh-change-lists branch from 24d1ccf to 2d6545c Compare July 28, 2026 17:59
@nickwolf
nickwolf marked this pull request as ready for review July 28, 2026 18:02
@nickwolf

Copy link
Copy Markdown
Author

Rebased onto main — the conflicts were just my copy of #102's commit colliding with the merged one, so dropping it resolved everything cleanly. No conflicts in the actual follow-up work, and your scheduling-days feature is untouched. Tests still green. Out of draft now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants